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(54) POSITION MEASURING SYSTEM AND DEVICE USING RSH EYE LENS 

(57)Abstract: 

PURPOSE: To measure a position by obtaining 
the cross point of a group of linear equation 
group corresponding to each target for each 
target, determining the three-dimensional 
position coordinates of each target, and then 
tracking a plurality of targets simultaneously. 
CONSTITUTION: An image pick-up device 3 
using a fish eye lens 1 is installed for 
determining light axis and then a system 
coordinate system (X, Y, Z) is determined 
arbitrarily at this point. Therefore, system 
coordinates are given to the fish eye images 
which are picked up by three image pick-up 
devices 3a. 3b, and 3c. Targets T1 , T2, T3,... 
within each image are subjected to image processing 6 and are detected and then an 
operation part 7 obtains three linear equations according to the light axis of the lens 1 
on the coordinate system (X, Y. Z). For example, three linear equations passing 
through the center of a visual field surface between the target T1 and the devices 3a. 
3b, and 3c are obtained. Namely, the number corresponding to the number of lenses 1 , 
namely each linear equation group consisting of three linear equations is obtained for 
the targets T1, T2, T3,.... The three-dimensional position coordinates are calculated 
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according to the cross points of the linear equation groups. 
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CLAIMS 



[Claim(s)] 

[Claim 1] With two or more image pick-up equipments which gave system system of 
coordinates to arbitration, determined an optical axis of each fish-eye lens to these 
system system of coordinates, and used said each fish-eye lens From each fish eye 
image given, an object which exists on this fish eye image, respectively is detected, 
and only a number corresponding to the number of said image pick-up equipment asks 
for a straight line equation passing through this object and a field-of-view side center 
of image pick-up equipment in said system system of coordinates, respectively. Two 
or more of these straight line equations A straight line equation group made into 1 set 



A positioning method using a fish-eye lens characterized by asking about said each 
detected object, respectively, asking for an intersection of a straight line equation 
group corresponding to said each object for every object, respectively, and 
determining a three-dimension position coordinate of each of said object in said 
system system of coordinates. 

[Claim 2] A positioning method using a fish-eye lens according to claim 1 
characterized by carrying out tracking processing of said location data which 
memorizes location data to time of day of each of said object in memory with that 
measurement time of day, and is memorized by this memory for every object, and 
searching for a track of an object, respectively. 

[Claim 3] Two or more fish-eye lenses which catch an object, and image pick-up 
equipment which picturizes a fish eye image obtained by this fish-eye lens. By the 
image-processing section which carries out the image processing of said fish eye 
image picturized by this image pick-up equipment, and detects said object, and an 
optical axis and said system system of coordinates of each of said fish-eye lens 
Operation part which computes a three-dimension position coordinate of said object 
detected in said image-processing section, Memory which memorizes a 
three-dimension position coordinate of each of said object computed by this 
operation part, a data processor for displaying in three dimensions a three-dimension 
position coordinate of each of said object memorized by this memory on monitor 
display, and a display which displays a three-dimension position coordinate of said 
object — since — positioning equipment using a fish-eye lens characterized by 
becoming. 

[Claim 4] the tracking processing section which carries out tracking processing of the 
three-dimension position coordinate of each of said object memorized by said 
memory, and determines a track of each of this object, a data processor for displaying 
a track of each of said object in three dimensions on monitor display, and a display 
which displays a track of said object — since — positioning equipment using a 
fish-eye lens according to claim 3 characterized by becoming. 
[Claim 5] Said at least three fish-eye lenses are positioning equipment using a 
fish-eye lens given [ respectively ] in claim 3 and claim 4 which are characterized by 
using. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the positioning method which can 
acquire the three-dimension positioning information on two or more objects using a 
fish-eye lens, and its equipment. 
[0002] 

[Description of the Prior Art] In the positioning equipment based on the present image 
information, an object is followed with the camera which usually has two sets of 
telephoto lenses, in this method, the direction of a camera (the direction of an optical 
axis of a lens) is adjusted so that an object may be obtained at the center of the visual 
field of each camera, and the location of an object is computed by trigonometry from 
the angle information on the installation location of two cameras, and an optical axis. 
Therefore, an adjusting device including the servo system for turning two cameras to 
an object correctly and the measurement-of-angle equipment which measures the 
direction of an optical axis of a camera correctly are required. 
[0003] 

[Problem(s) to be Solved by the Invention] Thus, when following an aim with the usual 
camera, responsibility with high servo system and measurement-of-angle system is 
required. However, it is quite difficult to position to coincidence, following enough with 
the present technology, when the passing speed of an object is quick. Moreover, in 
order only for one object to be able to follow with one equipment in such a method but 
to follow two or more two or more objects, the equipment of only the aim significant 
work is required as a system. Thus, with one equipment, two or more objects were not 
able to be followed to coincidence. 
[0004] 

[Means for Solving the Problem] With two or more image pick-up equipments which 
this invention gave system system of coordinates to arbitration, determined an optical 
axis of each fish-eye lens to these system system of coordinates, and used each of 
this fish-eye lens From each fish eye image given, an object which exists on this fish 
eye image, respectively is detected, and only a number corresponding to the number 
of this image pick-up equipment asks for a straight line equation passing through this 
object and a field-of-view side center of image pick-up equipment in system system 
of coordinates, respectively. Two or more of these straight line equations Ask, 
respectively about each object which had a straight line equation group made into 1 



set detected, ask for an intersection of a straight line equation group corresponding to 
each object for every object, respectively, and a three-dimension position coordinate 
of each object in system system of coordinates is determined. It follows to 
coincidence and two or more objects are positioned. 
[0005] 

[Function] About the object caught by the fish eye image, while only the number 
corresponding to each object asks for the straight line equation group which made 1 
set the straight line equation of the number corresponding to the number of image 
pick-up equipment which used the fish-eye lens, it asks for the intersection of this 
straight line equation group for every object, and the positioning information on an 
object is acquired in quest of the three-dimension position coordinate of this 
intersection. 
[0006] 

[Example] The example of this invention is explained to details based on drawing 1 - 
drawing 7 . The fish eye image according [ drawing 2 ] to a fish-eye lens with an angle 
of visibility of 1 80 degrees and the fish eye image according [ drawing 3 ] to three 
fish-eye lenses according [ drawing 1 ] to the important section block diagram of this 
invention, positioning information, and drawing 4 are explanatory drawing for 
positioning, and explanatory drawing in which drawing 5 - drawing 7 show the example 
of arrangement of a fish-eye lens. In drawing 1 - drawing 3,1 is a fish-eye lens, it is 
combined with the optoelectric transducers 2, such as CCD, image pick-up equipment 
3 is constituted, and three image pick-up equipments 3a, 3b, and 3c are used in this 
example. By installing the image pick-up equipment 3 using a fish-eye lens 1 , as 
shown in drawing 2 and drawing 3 , an optical axis L (La, Lb, and Lc) is determined. 
Moreover, the system system of coordinates (X, Y, Z) of a system are determined as 
arbitration, when a fish-eye lens 1 is installed, therefore, three image pick-up 
equipment 3a, 3b, and 3c each which was picturized ~ fish eye image 4a, 4b, and 4c A 
system coordinate is given by (Xb, Yb, Zb), and (Xc, Yc and Zc), respectively (Xa, Ya, 
and Za), as shown in drawing 3 . 

[0007] 5 is an interface, and in order to carry out the image processing of the fish eye 
image 4 picturized with image pick-up equipment 3, digital conversion of it is carried 
out. 6 is the image-processing section, and the image processing of the object T1 in 
the fish eye image 4, T2, and T3 ... is carried out, and it is detected. 7 is operation part 
and three straight line equations are called for with the optical axis L of the fish-eye 
lens 1 on system system of coordinates (X, Y, Z). For example, object Tl Each image 
pick-up equipment 3a, and 3b and 3c Three straight line equations passing through a 
field-of^view side center are called for, respectively. That is, the straight line equation 



group which makes the straight line equation of the number corresponding to the 
number of a fish^eye lens 1 1 set (it is 1 set with three straight line equations at this 
example) is called for about each object T1 . T2, and T3 respectively, and a 
three-dimension position coordinate is computed from the intersection of this straight 
line equation group. 8 is memory, 9 is the tracking processing section, tracking 
processing of the fish eye image 4 is carried out serially, and the track of an object T1 , 
T2, and T3 .. is determined. 1 0 is a data processor for an image output, the track of an 
object T1 , T2, and T3 ... is changed into a video signal, and image data processing for 
displaying in three dimensions on the monitor display of an indicating equipment 11 is 
performed. 12 is the Records Department and the fish eye image 4 picturized with 
image pick-up equipment 2 is recorded temporarily. 

[0008] Next, a positioning principle is explained. As shown in drawing 2 , while the 
angle of visibility of a fish-eye lens 1 is generally 1 80 degrees, an optical axis L is a 
perpendicular passing through the center of an image side, and the straight line 
equation in these system system of coordinates is determined by the installation of a 
fish-eye lens 1 (the image pick-up equipment 3 using a fish-eye lens 1 is shown). The 
fish eye image 4 made with the fish-eye lens 1 with an angle of visibility of 180 
degrees is circular, the center O of the fish eye image 4 when seeing the object T1 
which exists ahead of a fish-eye lens 1 , T2, and T3 .. shows the front (the direction of 
optical-axis L) of a fish-eye lens 1 , and, as for C of the point of 90 degrees of 
longitudinal directions of a fish-eye lens 1, and the vertical direction, and D point, the 
longitudinal direction A and the B point show right above a fish-eye lens 1 and right 
under, respectively. In addition, system system of coordinates (X, Y, Z) are determined 
as arbitration, when a fish-eye lens 1 is installed. Moreover, since the configuration of 
an object T1, T2, and T3 ... is distorted in accordance with the circumference of the 
fish eye image 4 as it separates from Center O when an object T1, T2, and T3 are 
seen with a fish-eye lens 1, it is difficult to recognize an object T1, T2, and T3 ... in a 
configuration, then, the light spot (stroboscope) which blinks to the empennage of the 
aircraft as a means by which this invention person recognizes the objects T1, such as 
an aircraft. T2, and T3 ... an object T1 , T2, and T3 — it set as ... When an object T1 , T2, 
and T3 ... are punctiforms, it is not distorted even if the object T is in the 
circumferencial direction of the fish eye image 4. 

[0009] Then, while determining the configuration of an object T1, T2, and T3 ... as a 
point (light spot), it sets to drawing 3 , and it is an object T1 and T2 as a punctiform 
object. It is set up if two pieces exist. Therefore, it is two objects T1 and T2 in the fish 
eye image 4 (4a, 4b, 4c) which two or more image pick-up equipments 3 (3a, 3b, 3c ...) 
which used the fish-eye lens 1 , respectively give. It exists and is this object T1 and T2. 



As common positioning information, by the installation of image pick-up equipment 3 
Respectively The installation coordinate of three image pick-up equipments 3a. 3b, 
and 3c (Xa, Ya, and Za), (Xb, Yb and Zb) and (Xc, Yc, and Zc) the direction La (thetaa 
and psia) of three opticals axis, Lb (thetab and psib), and Lc (thetac and psic) are 
determined, respectively. 

[0010] Next, object T1 As related positioning information The object T1 when seeing 
from 1 St image pick-up equipment 3a Object T1 when seeing from the displacement 
angle (thetaal, psial) from [ La ] ****** (thetaa and psia). and 2nd image pick-up 
equipment 3b The object T1 when seeing from the displacement angle (thetabi, psibi) 
from [ Lb ] an optical axis (thetab and psib), and 3rd image pick-up equipment 3c The 
displacement angle (thetaci, psicl) from [ Lc ] ****** (thetac and psic) is acquired. 
Similarly, it is an object T2. As related positioning information The object T2 when 
seeing from 1st image pick-up equipment 3a Object T2 when seeing from the 
displacement angle (thetaa2, psia2) from [ La ] ****** (thetaa and psia), and 2nd 
image pick-up equipment 3b The object T2 when seeing from the displacement angle 
(thetab2, psib2) from [ Lb ] an optical axis (thetab and psib), and 3rd image pick-up 
equipment 3c The displacement angle (thetac2. psic2) from [ Lc ] ****** (thetac and 
psic) is acquired. 

[001 1] In system system of coordinates Thus, the straight line equation of the optical 
axis L of a fish-eye lens 1 (it depends in an installation location and the direction L of 
an optical axis). The object T1 identified on this fish eye image 4, and T2 From location 
data, the straight line equation in the three-dimensions space which ties all the 
objects T1 on this fish eye image 4 and T2 ... and image pick-up equipment 3 is given 
about the fish eye image 4 obtained by one fish-eye lens 1 . As shown in drawing 1 and 
drawing 4 , with then, three image pick-up equipments 3 (3a. 3b, 3c) installed in three 
mutually different places It is two objects T1 and T2 on three fish eye images 4 (4a, 4b. 
and 4c) obtained, respectively. If the location of Object T (T1 and T2) is read in image 
information when it exists One object Tl Three straight line equation fla, fib, and fic 
which connect the field-of-view side center of three image pick-up equipments 3a, 3b. 
and 3c, respectively are obtained. Object T2 Even if it attaches, it is an object T2 
similarly. Three straight line equation f2a and f2bs, and f2c which connect the 
field-of-view side center of three image pick-up equipment 3a, 3b, and 3c, 
respectively are obtained, thus, the straight line equation group which makes 1 set the 
straight line equation of the number corresponding to the number of image pick-up 
equipment 3 — each object Tl and T2 every — that is. 2 sets is obtained. Thus, since 
Object T is located in the intersection at which the straight line of the number 
corresponding to the number of the image pick-up equipment 3 which 1 set of 



obtained straight line equation groups give crossed one place, if this intersection is 
computed, this point will give the position coordinate of the object T in 
three-dimension space. 

[0012] Next, based on the above and a positioning principle, actuation of the 
positioning equipment for actually acquiring the positioning information on Object T is 
explained based on drawing 1 and drawing 5. First, about the case where two or more 
objects T exist, in order to abolish a false image completely, each object T must be 
caught by the visual field of at least three image pick-up equipments 3. then, each of 
three image pick~up equipments 3 (3a, 3b, 3c) — when positioning the object T in the 
visual field of 180 degrees of front, as shown in drawing 5 , in this example, three image 
pick-up equipments 3 (3a, 3b, 3c) are installed so that an optical axis L (La, Lb, and 
Lc) may become parallel on the same plane, respectively. However, the direction L of 
an optical axis is max, and the angular resolution of a fish-eye lens 1 falls as it is left 
after this. Moreover, when Object T is on the straight line which ties two image 
pick-up equipments 3 on the installation plane of image pick-up equipment 3, it may 
double and forgery may occur. 

[0013] Therefore, when positioning Object T. it is necessary to double with the 
purpose referred to as which field to position in what precision, and to rationalize 
arrangement of image pick-up equipment 3. For example, when you do not need the 
angle of visibility of 1 80 degrees, as it shows in drawing 6 , they are the optical axis La 
of three image pick-up equipments 3, Lb, and Lc. By installing so that it may cross 
ahead, it is possible to raise the positioning precision in the longitudinal direction of a 
front field larger than the case where reduce the positioning precision of the front of 
2nd image pick-up equipment 3b, and it is shown in drawing 5 . Moreover, if five image 
pick-up equipments 3b, 3c, 3d, and 3e are arranged in three dimensions in total in the 
direction of four directions focusing on image pick-up equipment 3a as shown in 
drawing 7 , in the vertical direction, the positioning precision over a larger field can be 
raised similarly. Furthermore, when the angle of visibility of 180 degrees or more is 
required, it can be made to improve by using two or more image pick-up equipment 
groups similarly combined about positioning precision again by arranging image 
pick-up equipment 3 in three dimensions like positive 6 face piece or positive 12 face 
piece. That is, a positioning field is determined by the physical relationship and the 
number of a fish-eye lens 1 (image pick-up equipment 3). 
[0014] Moreover, when the visual field image which a fish-eye lens 1 makes is 
sufficiently large, at 400,000 elements or the time, 1 million or more things are possible 
also for use as an element number of an optoelectric transducer 2 (it is hereafter 
described as CCD2), and it is also possible to increase the element number of CCD2, 



to measure improvement in resolution, to lessen the use number of image pick-up 
equipment, and to improve positioning precision. While the above requirements are 
examined, it opts for the number of the image pick-up equipment 3 using a fish-eye 
lens 1 , and arrangement from a positioning angle of visibility, a positioning field, 
positioning precision, etc. Decision of arrangement of image pick-up equipment 3 
determines the optical axis L of each fish-eye lens 1 . That is, the direction of a normal 
of a fish-eye lens 1 is an optical axis L With it, the description as a straight line 
equation of the three dimension in these system system of coordinates is made. 
[0015] Next, as shown in drawing 3 , three fish eye images 4 (4a, 4b, and 4c) are 
obtained from three image pick-up equipments 3 (3a, 3b, 3c), respectively. Photo 
electric conversion of this fish eye image 4 is carried out by CCD2, respectively, and it 
is changed into a video signal. In addition, when positioning information on Object T 
does not need to be searched for in the real time (for example, when following a track 
etc.), this video signal is stored temporarily at the Records Department 12, and is 
behind processed in the image-processing section 6. The video signal of each fish eye 
image 4 in CCD2 is changed into a digital signal through an interface 5, and is inputted 
into the image-processing section 6. In the image-processing section 6, the image 
processing for detecting Object T is made, the object T picturized by each fish eye 
image 4 is detected, and this object T is detected with the what times temporal horn 
when which has shifted to the optical axis L, and a right-and-left **** degree. That 
image information (Xa1, Yal), (Xa2, Ya2) It is indicated by two-dimensional like 
(Xbl. Ybl), .... (Xb2, Yb2), (Xcl, Yd), and ... (Xc2, Yc2), and is inputted into operation 
part 7. 

[0016] In operation part 7, while the equation of an optical axis L is determined on the 
basis of system system of coordinates, it passes along this optical axis L, and the 
straight line which crosses Object T through the central point O of the fish-eye lens 1 
in which the field-of-view side center of image pick-up equipment 3 is shown (XO, YO, 
and ZO) is called for. The equation of this straight line (x-xOa)/alphaa = (y-yOa) /betaa 
= The straight line equation group which 1 set expressed with 
z-20a/gammaa(x-xOb)/alphab =(y-yOb)/betab =(z-zOb)/gammab(x-xOc)/alphac 
=(y-yOc)/betac =(z-zOc)/gammac becomes from three straight line equations is 
called for only for the number of Objects T. Thus, 2 sets of objects T (T1 and T2) are 
called for for the straight line equation group which makes 1 set the straight line 
equation of the number corresponding to the number of a fish-eye lens 1 , respectively. 
Next, the intersection of 2 sets of these straight line equation groups is computed by 
operation part 7, respectively, and the list of intersections is obtained. Only the point 
(point with the same output of the number corresponding to the number of a fish-eye 



lens 1) that there is the same output of three pieces is extracted from the list of this 
intersection, this point expresses the intersection of three straight line equations, i.e., 
a straight line equation group, and this intersection gives the location data of Object T. 
In addition, when the number of a fish-eye lens 1 is three pieces, it generates as an 
intersection of two straight line equations, and a false image is not generated at the 
intersection of three straight line equations. Therefore, if the number of fish-eye 
lenses 1 increases, the probability for a false image to occur indeed will become small. 
[001 7] Thus, the location of the intersection when the straight line of the number 
corresponding to the number of a fish-eye lens 1 crosses at one point the same [ of 
the straight line equation of the number corresponding to the number of the fish-eye 
lens 1 which constitutes a straight line equation group ] is computed as 
three-dimension space coordinates (Xn, Yn, Zn, and tm) of the object in system 
system of coordinates, and, as for this value, the three-dimension position coordinate 
is shown. However, t shows time of day. That is, N sets of straight line equation groups 
which make 1 set three straight line equations obtained from three image pick-up 
equipments 3 when the number of Objects T is N individual are called for, and, for the 
straight line equation group of this N group, a ***5K3|e*** part is max (3N-1) mutually! In 
a part an intersection Although connected, the intersection at which three straight 
lines cross is only N part equivalent to the number of Objects T. For example, the 
straight line equation obtained from three image pick-up equipments 3 supposing the 
number of the objects T to position was ten is 29! Although the intersection of a part 
is connected, the intersection (Object T serves as an intersection) at which three 
straight line equations cross is ten equivalent to the number of Objects T. 
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It comes out, the three-dimension position coordinate shown is searched for from 
time of day t1, t2, and t3 respectively, and this three-dimension position coordinate 
is memorized by memory 8. 

[0018] In the tracking processing section 9, tracking processing of the data 
memorized by memory 8 is carried out about time of day t1 and t2 and t3 namely, 
the locus of each coordinate is called for, and the track of Object T is acquired in the 
image seen from the top. Since it is the coordinate data in the continuous time of day 
by which three-dimension positioning was carried out, to the space-coordinates 
system of arbitration, visualization processing is possible for this track as a 
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three-dimensionHocus, the processing for displaying in three dimensions on monitor 
display with a data processor 1 0 is made, and it is changed into a video signal, and is 
displayed on a display 1 1 as a continuous image, and a knowledge based system is 
done by the controller etc. 
[0019] 

[EfFect of the Invention] With two or more image pick-up equipments which this 
invention gave system system of coordinates to arbitration, determined the optical 
axis of each fish-eye lens to these system system of coordinates, and used each of 
this fish-eye lens From each fish eye image given, the object which exists on this fish 
eye image, respectively is detected, and only the number corresponding to the number 
of this image pick-up equipment asks for the straight line equation passing through 
this object and the field-of^view side center of image pick-up equipment in system 
system of coordinates, respectively. Two or more of these straight line equations 
Since it was alike so that it might ask, respectively about each object which had the 
straight line equation group made into 1 set detected, it might ask for the intersection 
of the straight line equation group corresponding to each object for every object, 
respectively and the three-dimension position coordinate of each object in system 
system of coordinates might be determined By one system, two or more objects can 
be positioned to coincidence, and it can follow. Moreover, the track of an object can 
be computed and analyzed if the image information of an object is memorized. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the example of this invention. 
[Drawing 2] It is the fish eye image which a fish-eye lens 1 gives. 
[Drawing 3] It is the fish eye image which shows the example of this invention and 
three fish-eye lenses 1 give. 

[Drawing 4] It is explanatory drawing showing the example of this invention. 
[Drawing 5] It is drawing in which showing the example of this invention and showing 
the example of arrangement of image pick-up equipment. 

[Drawing 6] It is drawing in which showing the example of this invention and showing 
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the example of arrangement of image pick-up equipment. 

[Drawing 7] It is drawing in which showing the example of this invention and showing 
the example of arrangement of image pick-up equipment. 
[Description of Notations] 

I ... Fish-eye lens 

3 ... Image pick-up equipment 

4 ... Fish eye image 

5 ... Interface 

6 ... Image-processing section 

7 ... Operation part 

8 ... Memory 

9 ... Tracking processing section 
10.. Data processor 

I I .. Display 

12.. Records Department 
T ... Object 
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[Drawing 4] 




[Drawing 5] 
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[Drawing 6] 
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